1. Field of the Invention
This invention relates to the fields of video compression and motion-compensated image interpolation for frame-rate conversion, and, more particularly, to an apparatus and methods for filtering motion vectors provided by block-based motion estimation (ME) apparatuses and methods.
2. Description of Related Art
Many image and video processing apparatuses and methods, such as motion-compensated image interpolation, rely on estimated motion vectors. Typically these motion vectors are block-based, in which a motion vector represents a rectangular group of pixels. In some apparatuses and methods, such as image interpolation, it is important that these motion vectors be ‘true’, that is substantially free of any errors, so as not to produce any processing artifacts in the output results. Motion estimation (ME) methods using exhaustive full-search block-matching techniques are prone to errors in the motion vector fields. These errors may come from different types of content in a source and are nearly impossible to eliminate without applying some extra processing steps on the motion vector fields. Looking at the motion vector field as a signed intensity image, errors can be easily detected as outliers, meaning a motion vector value is different than most of the neighboring blocks' motion vector values. Motion Vector Filtering (MVF) may remove these outliers and provide a smoother, more consistent motion vector field. Some outliers are more isolated and easier to detect and filter out than others. Some motion vector errors can occur in groups where all neighboring motion vectors are similar, but all erroneous. The classic example of when this may occur is in areas of periodic structures, also referred to as lattice structures. For example, a lattice structure having a period of 8 pixels and also a true motion of 8 pixels may easily be estimated as having a motion of 0, causing “breaking” artifacts in an image interpolation scheme. These regions within the motion vector field are harder to filter out since the errors show up as clusters, making them harder to detect as errors.
U.S. Pat. No. 5,148,269 describes a device in which a small group of coarse (block-based) motion vectors is filtered to produce a finer motion vector, using the median value of an input subset. The center block is first partitioned into 4 sub-blocks. For each sub-block, a median value of the center block's motion vector component value and the same component values of the two blocks neighboring the sub-block is determined. The median value is assigned to the sub-block if a couple of simple criteria are met. The operation is then repeated until a motion vector is assigned to each pixel. This method was intended to reduce the dirty window effect, in which block-based motion vectors were used directly to generate the interpolated image. The method also has a smoothing effect on the motion vector field and provides the ability to remove isolated motion vectors. However, the method is “blind” as it operates on a set of motion vectors without verifying the quality of the replacement vector (by using error metrics that are commonly used in motion estimation, block-matching algorithms). The method also lacks the use of the source image as a guide for locating object boundaries, or the use of any information with regard to periodic structures that may be present.
What is needed is a method and an apparatus to replace erroneous motion vectors in a motion estimation apparatus that may have been detected as isolated from its neighbors or as a small outlier group, and may appear to be erroneous on a closer analysis.